black hat USA2012 

Scientific but Not Academical Overview of Malware 
Anti-Debugging, Anti-Disassembly and Anti-VM 

Technologies 


0 Qualys 

Rodrigo Rubira Branco (@BSDaemon) 
Gabriel Negreira Barbosa (@gabrielnb) 
Pedro Drimel Neto (@pdrimel) 



Were we live... 





0 QUALYS" 


BLACK HAT USA 2012 





Nah, we live actually here... 



0 Qualys 


BLACK HAT USA 2012 








Sao Paulo 







0 Qualys 


BLACK HAT USA 2012 







Agenda 


. Introduction / Motivation 
. Objectives 
. Methodology 
. Dissect || PE Project 
. Executive Summary 
. Techniques 
. Resources 
. Conclusions 


0 QUALYS" 


BLACK HAT USA 2012 



Introduction / Motivation 


. Hundreds of thousands of new samples every week 

. Still, automation is about single tasks or single analysis 

. Presentations still pointing tens of thousands in tests (what 
about the millions of samples?) 

. Companies promote research which uses words such as 
‘many’ instead of X number 
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Before continue, some definitions ... 


. Anti-Debugging 

. Techniques to compromise debuggers and/or the debugging process 

. Anti-Disassembly 

• Techniques to compromise disassemblers and/or the disassembling process 

. Obfuscation 

. Techniques to make the signatures creation more difficult and the disassembled 
code harder to be analyzed by a professional 

. Anti-VM: 

. Techniques to detect and/or compromise virtual machines 
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Objectives 


. Analyze millions of malware samples 
. Share the current results related to: 

. Anti-Debugging 
. Anti-Disassembly 
. Obfuscation 
. Anti-VM 

. Keep sharing more and better results in our portal 

( ): 

. New malware samples are always being analyzed 
. Detection algorithms are constantly being improved 

. The system does not analyze only anti-RE things 
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Dissect 


PE Project 


I 

. Scalable and flexible automated malware analysis system 

. Receives malware from trusted partners 

. Portal available for partners, researchers and general media 
with analysis data 
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Dissect 11 PE-Overview 


. Free research malware analysis system for the community 

. Open architecture 
. Works with plugins 

. 10 dedicated machines distributed in 3 sites: 

• 2 sites in Brazil (Sao Paulo and Bauru cities) 

• 1 site in Germany 

. Some numbers: 

. Receives more than 150 GB of malwares per day 
. More than 30 million unique samples 
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Dissect 


Coogle 
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Dissect 11 PE-Backend 


. Each backend downloads samples scheduled for analysis 
(our scheduler algorithms are documented in a IEEE 
Malware2011 paper) 

. Analyze samples 

• Both static and dynamic analysis currently supported 

. Analysis results accessible from the portal 

. Sync'ed back from the backend 

. Some characteristics: 

• Plugins 

• Network traffic 

• Unpacked version of the malware 
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Dissect 11 PE - Plugins 


. Samples are analyzed by independent applications named 
“plugins” 

. Easy to add and/or remove plugins 

• Just a matter of copy and remove their files 

. Language independent 
. Easy to write new plugins: 

. Needed information come as arguments 

- We usually create handlers so the researcher does not need to change his actual 
code 

. Simply print the result to stdout 

- The backend takes care of parsing it accordingly 
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Dissect 


PE-Plugi 


I 

. Python 
print “My plugin result.” 

. C 

#include <stdio.h> 
int main(int argc, char 

printf(“My plugin resultAn”); 
return 1; 

} 
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Dissect 


PE - Plugin Types 



. Static: 

. Usually executed outside of the VM (we already have an exception for the 
unpacking plugin) 

. Failsafe: errors do not compromise the system 

. Might get executed in one of two different situations depending on where we 
copied the plugin: 

- Before the malware is executed 

- After the malware was executed 

. Dynamic: 

. Executed inside a Windows system (for now the only supported OS, soon others) 
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Dissect 


PE - Network Traffic 


I 

. During dynamic analysis all the network traffic is captured 
. Pcap available at the portal 
. Dissectors: 

. Analyze the pcap and print the contents in a user-friendly way 
. Supporting IRC, P2P, HTTP, DNS and other protocols 
. SSL inspection (pre-loaded keys) 
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Methodology 


. Used a total of 72 cores and 100 GB of memory 
. Analyzed only 32-bit PE samples 
. Packed samples: 

• Different samples using the same packer were counted as 1 unique 
sample 

- So, each sample was analyzed once 

• Analyzed all packers present among the 4 million samples 

. Unpacked samples: 

. Avoided samples bigger than 3,9 MB for performance reasons (with some 
exceptions such as the Flame Malware) 
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Methodology 


. Static analysis: 

. Main focus of this presentation 
. Improves the throughput (with well-written code) 

. Not detectable by malwares 

. Dynamic counter-part: 

. It is not viable to statically detect everything 

. Already developed and deployed, but is not covered by this presentation 
- The related results can be found at https://www.dissect.pe 
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Methodology 


. Malware protection techniques in this work: 

. State-of-the-art papers/journals 
. Malwares in the wild 

. Some techniques we documented are not yet covered by our system: 

- The system is constantly being updated 

. All techniques were implemented even when there were no public examples of it 
(github) 

. Our testbed comprises 883 samples to: 

. Detect bugs 
. Performance measurement 
. Technique coverage 
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Methodology 


. Possible techniques detection results: 

• Detected: 

- Current detection algorithms detected the malware protection technique 

• Not detected: 

- Current detection algorithms did not detect the malware protection technique 

• Evidence detected: 

- Current detection algorithms could not deterministically detect the protection 
technique, but some evidences were found 
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Methodology 


. Analysis rely on executable sections and in the entrypoint 
one 

. Decreases the probability to analyze data as code 
. Improves even more the analysis time 

. For now we miss non-executable areas, even if they are referred by analyzed 
sections (future work will cover this) 

. Disassembly-related analysis framework: 

. Facilitates the development of disassembly analysis code 
. Speeds up the disassembly process for plugins 
. Calls-back the plugins for specific instruction types 
. Disassembly once, analyze all 
. Care must be taken to avoid disassembly attacks 
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Executive Summary 


o 
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Packed vs Not Packed 
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Top Packers 


o 



Top 10 Packers 

Others 

■ 3,78 

MaskPEV20yzkzero 

0,24 

WiselnstallerStub 

0,25 

ASProtect 

0,30 

ProtectSharewareVlleCompservCMS 

1 0,81 

ASPack 

1 1,08 

BobSoftMi ni Delphi BoB BobSoft 

1 1,12 

PE Com pact 

■ 2,36 

Armadillo 

6,87 

UPX 

16,12 

unknown/none 





0 QUALYS" 


BLACK HAT USA 2012 





Malware Targeting Brazilian Banks 
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Protecting Mechanisms of Packers 


Paper (yes, we wrote one...) 
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Protected Samples 



Unprotected 
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Anti-RE Categories 


Anti-Disassembly 


Anti-Debugging 



Obfuscation 


Anti-VM 
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43 , 21 % 


68 , 95 % 


81 , 40 % 


BLACK HAT USA 2012 






Anti-Disassembly 
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Anti-Debugging 



Anti-Debugging 


Anti-Hook 
Header Entrypoint 
Heap Flags 
Hardware Breakpoint 
SoftICE Int 1 
PEB NtGlobalFlag 
Software Breakpoint 
SS register 

IsDebuggerPresent PEB BeingDebugged 



71 , 72 % 
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Obfuscation 



Obfuscation 


Function Call Obfuscation 

Stealth Import Win API 
PEB->Ldr Address Resolving 
NOP Sequence 
Push Pop Math 
Instruction Substitution (Push Ret) 
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Anti-VM 


SIDT 

SGDT 


VPC Invalid Opcode 


STR 

SLDT 

IN 


0,06% 

0,06% 

I 2,14% 
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19,91% 

l ^ 17 ^ 
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Anti-Debugging Techniques 


. Studied and documented 33 techniques 
. Currently scanning samples for 30 techniques 

. Detected: Marked in green 
. Evidence: Marked in yellow 
. Not covered: Marked in black 
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Anti-Debugging Techniques 


o 


. PEB NtGlobalFlag (Section 3.1) 

. IsDebuggerPresent (Section 3.2) 

. CheckRemoteDebuggerPresent (Section 3.3) 

. Heap flags (Section 3.4) 

. NtQuerylnformationProcess - ProcessDebugPort (Section 3.5) 

. Debug Objects - ProcessDebugObjectHandle Class (Section 3.6) 

. Debug Objects - ProcessDebugFlags Class [1] (Section 3.7) 

. NtQuerySystemlnformation - SystemKernelDebuggerlnformation (Section 3.8) 
. OpenProcess - SeDebugPrivilege (Section 3.9) 

. Alternative Desktop (Section 3.10) 
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Anti-Debugging Techniques 


. Self-Debugging (Section 3.11) 

. RtlQueryProcessDebuglnformation (Section 3.12) 
. Hardware Breakpoints (Section 3.13) 

. OutputDebugString (Section 3.14) 

. Blocklnput (Section 3.15) 

. Parent Process (Section 3.16) 

. Device Names (Section 3.17) 

. OllyDbg - OutputDebugString (Section 3.18) 

. FindWindow (Section 3.19) 

. SuspendThread (Section 3.20) 


o 
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Anti-Debugging Techniques 


. SoftlCE - Interrupt 1 (Section 3.21) 

. SS register (Section 3.22) 

. UnhandledExceptionFilter (Section 3.23) 

. Guard Pages (Section 3.24) 

. Execution Timing (Section 3.25) 

. Software Breakpoint Detection (Section 3.26) 
. Thread Hiding (Section 3.27) 

. NtSetDebugFilterState (Section 3.28) 

. Instruction Counting (Section 3.29) 

. Header Entrypoint (Section 3.30) 

. Self-Execution (Section 3.31) 
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Anti-Disassembly Techniques 


. Studied and documented 9 techniques and variations 
. Currently scanning samples for 8 techniques and variations 

. Detected: Marked in green 
. Evidence: Marked in yellow 
. Not covered: Marked in black 
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Anti-Disassembly Techniques 


. Garbage Bytes (Section 4.2.1) 

. Program Control Flow Change (Section 4.2.2) 

. Direct approach 
. Indirect approach 

. Fake Conditional Jumps (Section 4.2.3) 

. XOR variation 
. STC variation 
. CLC variation 
. Call Trick (Section 4.2.4) 

. Flow Redirection to the Middle of an Instruction (Section 4.2.5) 
. Redirection into other instructions 
. Redirection into itself 
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Obfuscation Techniques 


. Studied and documented 14 techniques and variations 

. Currently scanning samples for 7 techniques and 
variations 

. Detected: Marked in green 
. Evidence: Marked in yellow 
. Not covered: Marked in black 
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Obfuscation Techniques 


. Push Pop Math (Section 4.3.1) 

. NOP Sequence (Section 4.3.2) 

. Instruction Substitution (Section 4.3.3) 

• JMP variation 

• MOV variation 

• XOR variation 

• JMP variation (Push Ret) 

. Code Transposition (Section 4.3.4) 

. Program control flow forcing variation 
. Independent instructions reordering variation 
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Obfuscation Techniques 


. Register Reassignment (Section 4.3.5) 

. Code Integration (Section 4.3.6) 

• Fake Code Insertion (Section 4.3.7) 

. PEB->Ldr Address Resolving (Section 4.3.8) 

. Stealth Import of the Windows API (Section 4.3.9) 
. Function Call Obfuscation (Section 4.3.10) 
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Anti-VM Techniques 


. Studied and documented 7 techniques and variations 

. Currently scanning samples for 6 techniques and 
variations 

. Detected: Marked in green 
. Evidence: Marked in yellow 
. Not covered: Marked in black 


0 QUALYS" 


BLACK HAT USA 2012 



Anti-VM Techniques 


. CPU Instructions Results Comparison (Section 5.1) 

. SIDT approach 
. SLDT approach 
. SGDT approach 
. STR approach 
. SMSW approach 

. VMWare - IN Instruction (Section 5.2) 

. VirtualPC - Invalid Instruction (Section 5.3) 
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New Techniques 


. We understand that malware is quickly evolving, thus there is a need for analysis to go at least 
as fast 

. SSEXY 

. SSE obfuscation tool released in Hack in The Box Amsterdam (17-20 of May) by Jurriaan 
Bremer 

• In June we already had a plugin to detect it 
. Flame 

• The industry positioned it as completely new, embedding a LUA interpreter for rapid 
development of new capabilities 

. We implemented a plugin for the detection of embedded LUA as soon as the news came 
out and we can TELL you that there is no other malware containing LUA 

- We do not have to assume it as we have analysis results 
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New Techniques 


. On July, 25, Morgan Marquis-Boire and Bill Marczak 
released a paper about the FinFisher Spy Kit. Their paper 
mention many protection techniques used by the code: 

. A piece of code for crashing OllyDBG 
. DbgBreakPoint Overwrite (Covered in Section 3.33) 

. IsDebuggerPresent (Covered in Section 3.2) 

. Thread Hiding (Covered in Section 3.27) 

. Debug Objects - ProcessDebugObjectHandle Class (Covered in Section 3.6) 
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Resources 


. Sample code for the different techniques we detect are 
available on github: 


- We will open the repository just after the conference 


. Updated versions of the paper and presentation are going to 
be available at: 
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Resources - Portal Demo 


. Portal URL: 

. Any interested researcher / contributor / journalist can have 
access to the portal (drop us an email or come to the Qualys 
booth) 

. We are constantly updating the statistics and developing/ 
improving analysis algorithms 
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Conclusions 


. We analyzed millions of malware samples and showed 
scientific results about their usage of protection techniques 

. There are more techniques to implement and some 
algorithms to improve 

. We still have a lot to do... and so do you! Help us! 

. The portal ( ) is always updated with new and 

better results: 

. More detection techniques 
. More analyzed samples 
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